Module 3: Statistical Applications of Least Squares

This week we will learn more about the applications of least squares methods in statistics. We will learn about how weighted least squares methods allow us to apply linear regression in more cases, including those with heteroscedastic errors. Then we will show how to fit data using interpolating functions, and autoregressive models for time series. This discussion will lead into feature engineerging. Finally, we will explore validation and regularization, which is our first example of multi-objective least squares.

Deliverables

Homework 2 will be due Sunday, February 23rd at midnight.

Learning Objectives

• Least Squares and Linear Regression
• Generalized Least Squares
• Recursive/Online Least Squares
• Regularization and Multi-Objective Least Squares

Readings

Chapters 13 of Introduction to Applied Linear Algebra

Chapter 14 (on classification methods) is interesting but we won’t cover it.

Sections 4.2 and 4.5 of Introduction to Algorithms for Data Mining and Machine Learning